Input method

ABSTRACT

A method comprises causing a user interface to provide an output directly or indirectly specifying at least one object, analyzing substantially real-time data from a camera to determine if the at least one object is located in the field of view of the camera, and, in response to determining that the at least one object is located in the field of view of the camera, causing an indication that the object has been located to be provided through the user interface.

FIELD

The invention relates to an input method.

BACKGROUND

Devices such as mobile devices are generally provided with input meanssuch as physical keys or touch screens in order to allow a user tocontrol operations of the device, for example to play games using thedevice.

SUMMARY

According to a first aspect of the present invention, there is provideda method comprising causing a user interface to provide an outputdirectly or indirectly specifying at least one object, analyzingsubstantially real-time data from a camera to determine if the at leastone object is located in the field of view of the camera, and, inresponse to determining that the at least one object is located in thefield of view of the camera, causing an indication that the object hasbeen located to be provided through the user interface.

The method may further comprise causing display of the substantiallyreal-time data from the camera.

The method may further comprise, in response to determining that the atleast one object is located in the field of view of the camera, causingdisplay of graphical information highlighting the at least one object.The graphical information may comprise the outline of a shape such as arectangle enclosing a major part of the at least one object.

The method may further comprise measuring an interval between providingthe output directly or indirectly specifying at least one object anddetermining that the at least one object is located in the field of viewof the camera, and providing a value calculated based on the measuredinterval.

The method may further comprise providing an output directly orindirectly specifying plural objects, analyzing the substantiallyreal-time data from the camera to determine if one of the plural objectsis located in the field of view of the camera, and, in response todetermining that one of the plural objects is located in the field ofview of the camera, causing an indication of which of the plural objectshas been located to be provided through the user interface. The methodmay further comprise determining subsequent operations of a processbased on which of the plural objects has been located. The method mayfurther comprise providing a value calculated based on which of theplural objects has been located.

The method may form part of a game.

According to a second aspect of the present invention, there is providedapparatus, comprising at least one processor, and at least one memoryincluding computer program code, the at least one memory and thecomputer program code configured to, with the at least one processor,cause the apparatus at least to perform causing a user interface toprovide an output directly or indirectly specifying at least one object,analyzing substantially real-time data from a camera to determine if theat least one object is located in the field of view of the camera, and,in response to determining that the at least one object is located inthe field of view of the camera, causing an indication that the objecthas been located to be provided through the user interface.

The at least one memory and the computer program code may be configuredto, with the at least one processor, cause the apparatus to performcausing display of the substantially real-time data from the camera.

The at least one memory and the computer program code may be configuredto, with the at least one processor, cause the apparatus to perform, inresponse to determining that the at least one object is located in thefield of view of the camera, causing display of graphical informationhighlighting the at least one object. The graphical information maycomprise the outline of a shape such as a rectangle enclosing a majorpart of the at least one object.

The at least one memory and the computer program code may be configuredto, with the at least one processor, cause the apparatus to performmeasuring an interval between providing the output directly orindirectly specifying at least one object and determining that the atleast one object is located in the field of view of the camera, andproviding a value calculated based on the measured interval.

The at least one memory and the computer program code may be configuredto, with the at least one processor, cause the apparatus to performproviding an output directly or indirectly specifying plural objects,analyzing the substantially real-time data from the camera to determineif one of the plural objects is located in the field of view of thecamera, and, in response to determining that one of the plural objectsis located in the field of view of the camera, causing an indication ofwhich of the plural objects has been located to be provided through theuser interface. The at least one memory and the computer program codemay be configured to, with the at least one processor, cause theapparatus to perform determining subsequent operations of a processbased on which of the plural objects has been located. The at least onememory and the computer program code may be configured to, with the atleast one processor, cause the apparatus to perform providing a valuecalculated based on which of the plural objects has been located.

The at least one memory and the computer program code may be configuredto, with the at least one processor, cause the apparatus to perform amethod which form part of a game.

The apparatus may be a device further comprising user interfacecircuitry and user interface software configured to facilitate usercontrol of at least some functions of the device though use of a displayand configured to respond to user inputs, and a display and displaycircuitry configured to display at least a portion of a user interfaceof the device, the display and display circuitry configured tofacilitate user control of at least some functions of the device.

According to a third aspect of the present invention, there is providedapparatus, comprising means for causing a user interface to provide anoutput directly or indirectly specifying at least one object, means foranalyzing substantially real-time data from a camera to determine if theat least one object is located in the field of view of the camera, andmeans for, in response to determining that the at least one object islocated in the field of view of the camera, causing an indication thatthe object has been located to be provided through the user interface.

According to a fourth aspect of the present invention, there is provideda non-transitory computer readable medium having stored thereoninstructions for causing computing apparatus to perform a methodcomprising causing a user interface to provide an output directly orindirectly specifying at least one object, analyzing substantiallyreal-time data from a camera to determine if the at least one object islocated in the field of view of the camera, and, in response todetermining that the at least one object is located in the field of viewof the camera, causing an indication that the object has been located tobe provided through the user interface.

According to a fifth aspect of the present invention, there is provideda computer program, optionally stored on a medium, comprisinginstructions for causing the computer apparatus to perform the methodaccording to the first aspect of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS

Embodiments of the invention will now be described, by way of exampleonly, with reference to the accompanying drawings, in which:

FIG. 1 shows a device in which example embodiments of the invention areimplemented;

FIG. 2 illustrates some aspects of the hardware configuration of thedevice shown in FIG. 1;

FIG. 3 illustrates some aspects of the software configuration of thedevice shown in FIG. 1;

FIG. 4 is a flow diagram illustrating operation of the device shown inFIG. 1; and

FIG. 5 shows exemplary screenshots displayed during operation of thedevice shown in FIG. 1.

DETAILED DESCRIPTION

Referring to FIG. 1, a device 100 is shown. The device 100 embodiesaspects of the invention and is not limiting on the scope of theinvention. The device 100 has a touch sensitive display, or touchscreen,102 for displaying content and for receiving tactile user inputs. Thedevice 100 also has hardware keys, including a home key 104 a and animage capture key 104 b, as well as a speaker 106 and a microphone 108.The device 100 has two cameras, a first camera 110 and a second camerawhich is located on the opposite side of the device 100 to thetouchscreen 102 and is not shown in FIG. 1. The device 100 may be amobile computer, mobile phone, PDA, internet tablet, smartphone or otherdevice.

Referring mainly to FIG. 2, some aspects of the hardware configurationof the device 100 will now be described. The device 100 includes acontroller 200 comprising one or more processors (not shown). Thecontroller 200 communicates with the other hardware components via asystem bus 202. Each other hardware component is connected to the systembus 202 either directly or via an interface (generally denoted by thenumeral 204).

The touch sensitive display 102 comprises a display part 205 and atactile interface part 206. The controller 200 receives inputs from thetactile interface 206 and controls the display shown on the display part205. The touch sensitive display is connected to the system bus 202 viaan interface 204 a. The first camera 110 and the second camera 208 arealso connected to the system bus 202 via an interface 204 b. Each camera110, 208 may comprise any image sensing technology such as acharge-coupled device (CCD) or an active pixel sensor such as acomplementary metal oxide semiconductor (CMOS) device. User input means210 are connected to the system bus 202 via an interface 204 c. The userinput means 210 include the keys 104 a, 104 b and the microphone 108, aswell as any other input means such as a QWERTY or numeric keypad or anaccelerometer. The speaker 106 is also connected to the system bus 202via an interface 204 d. The device 100 also includes a transceiver 210connected to the system bus 202 via an interface 204 e for communicatingover a wireless link, such as a GSM, CDMA, UMTS, LTE, WiMax or Wi-Filink. The device 100 also has a rechargeable battery 212 for storingelectrical energy. Other hardware components (not shown) may also beincluded, such as a global positioning or system (GPS) or otherpositioning receiver and a vibrating element.

The device 100 includes both volatile memory 214 and non-volatile (ornon-transitory) memory 216. Both types of memory are accessed by thecontroller 200 via the system bus 202 and both types are connecteddirectly to the system bus 202. The volatile memory 214 is RAM, DRAM orany other type of volatile memory. The volatile memory 214 is used bythe controller 200 for the temporary storage of data, for instance whencontrolling the operation of another hardware or software component ormoving data between components. The non-volatile memory 216 is ROM,flash memory, a hard disk or any other type of non-volatile memory. Thenon-volatile memory 216 stores computer readable instructions 218 usedin operating the device 100 as well as storing content and personal data(not shown).

Referring mainly to FIG. 3, some aspects of the software configurationof the device 100 will now be described. The non-volatile memory 216stores an operating system 300, user interface software 302, an inputmodule 304 and various software applications (generally denoted by thenumeral 306).

The operating system 300 includes, amongst other things, a number ofsoftware modules that control operation of the device 100. Theapplications 306 and the input software 304 run on the operating system300. The user interface software 302 operates along with user interfacehardware, including the touchscreen 102, the keys 104 a, 104 b, thespeaker 106 and the microphone 108, thus forming a user interface. Theuser interface software 302 may be separate from the operating system300, in which case it interacts closely with the operating system 300,or may be integrated with the operating system 300.

As will be described in more detail below, the application 306 and theinput module 304 are configured, when executed by the controller 200using the memory 214, 216, to cause the device 100 at least to cause theuser interface to provide an output directly or indirectly specifying atleast one object, analyze substantially real-time data from the camera110, 208 to determine if the at least one object is located in the fieldof view of the camera; and in response to determining that the at leastone object is located in the field of view of the camera 110, 207, causean indication that the object has been located to be provided throughthe user interface.

The input module 304 includes an applications interfacing module 308 forreceiving messages from the applications 306, such as the request tolocate the specified object(s) in the field of view of the camera 110,208. The applications interfacing module 308 is also for sendingmessages to the applications 306, such as the indication that thespecified object(s) has been located. The input module 304 also includesa camera interfacing module 310 for receiving the image data from thecamera 110, 208 and, amongst other things, passing this image data to ananalyzing module 312 also included in the input module 304. Theanalyzing module 312 is for analyzing the image data and employingobject recognition methods to recognize the specified object(s). A mainmodule 314 is included in the input module 304 for directing some of theoperations of the other component modules, e.g., for determining thatthe request to locate a specified object has been received and fordirecting the analyzing module 312 to determine if the specifiedobject(s) is located in the field of view of the camera 110, 208. Themain module 314 is also connected to the user interface (UI) software302 via a UI software interfacing module 316 for receiving user inputsor providing outputs, such as displaying image data from the camera 110,208 and other graphical information on the touchscreen 102. In someexample embodiments, the main module 314 is also able to connect to anobject database 318 for receiving object data to be used for objectrecognition. In some example embodiments, some of the modules describedabove as being included in the input module 304 may be included in othersoftware components (not shown) instead or may be shared with othersoftware components.

Three software applications 306, namely a first application 306 a, asecond application 306 b and an internet browser application (‘browser’)306 c, are shown as being stored in the non-volatile memory 216 in FIG.3. However, it will be appreciated that there may be fewer or there maybe more applications 306. The first and second applications 306 a, 306 bmay be any type of application. For example, they may be games,educational applications, tools, etc. The browser 306 c is for allowinga user to browse web pages retrieved, for example, via the Internet bymeans of the transceiver 210. In some example embodiments, there may beno browser or there may be more than one browser. The applications 306can be installed in the non-volatile memory 216 before the device 100 issupplied to an end user or, for example, can be downloaded via theInternet and installed by the end user.

Applications 306 of various different types can make use of the inputmodule 304 in various different ways. For example, the application 306may be a game in which, as a part of the game, the user is challenged tolocate a particular object or particular objects as quickly as possibleand is given a score which depends partly or fully upon the time takento locate the object(s). The user may also be challenged to find one ofa set of different objects and be given a score which depends partly orfully upon which particular one of the set of objects has been located.Applications 306 of various different types can use the input module 304to enable a user to choose between one of a plurality of differentoptions by locating a particular object associated with a particularoption. In other words, subsequent operations of the application 306 aredetermined based upon the object which the user locates in the field ofview of the camera 110, 208. Of course, these different options need notnecessarily be explicitly made known to a user.

The input module 304 can also be used to determine the position of auser relatively precisely. For example, the user can be requested tolocate a specified object such as a building or landmark and, based ondetails about the located object (e.g., aspect, size, inclination), candetermine the position of the user in relation to the object. Suchinformation can be used in combination with GPS or other positioninginformation, or user-input information. Based on the user's position,the application 306 can direct the user to take various actionsincluding, for example, moving closer to the object or turning to faceanother direction.

Request can be made to locate practically any type of object. Exemplaryobjects include everyday items, buildings and structures, people,animals, and plants, as well as pictures, symbols or words includingthose included in artwork, books or other media. The objects to belocated may also vary depending upon the history of operation of theapplication 306, previous inputs by the user or by another person orapplication in communication with the device 100, previous objectslocated by the user, the time and date, the geographical location of thephone, and so forth.

Referring mainly to FIGS. 4 and 5, an input method according to anexample embodiment will now be described. It will be appreciated that,in these operations, the controller 200 causes the device 100 to performthe stated functions and actions under control of instructions that formpart of the user interface module 302, the input module 304 and/or theapplications 306 running on the operating system 300 of the device 100.During execution, some or all of the computer programs may be stored inthe volatile memory 214. In the event that only some of this computerprogram is stored in the volatile memory 214, the remainder resides inthe non-volatile memory 216. When the application is the browser 306 c,the device 100 operates in a similar way except that, in this case,scripting language or markup language associated with a web page willcause the browser 306 c to make use of the input method.

Firstly, at steps S400 and S401, the running application 306 (e.g. agame application) provides instructions to the user via the userinterface software 302 and, for example, the touchscreen 102 or thespeaker 106. The instructions identify one or more objects and requestthat the user photographs one of the objects. The application 306simultaneously or subsequently provides a request to locate a specifiedobject to the input module 304 (see step S402). In some exampleembodiments, the request specifies a set of different objects and thatany object in the set of different objects is to be located. In someexample embodiments, all of the objects in a set of different objectsare to be located in the field of view of the camera 110, 208. As shownin FIG. 5A, these instructions may describe the objects to be located.However, as mentioned above, the instructions to the user do notnecessarily have to specify which object(s) are to be located (althoughthe request sent to the input module 304 does); the instructions to theuser may contain a clue (e.g. a cryptic clue) as to the identity of theobject that is required to be located.

The request to locate the specified object(s) can include the datarequired by the input module 304 to determine whether or not a specifiedobject has been located in the field of view of the camera 110, 208.Such data is referred to herein as object data. The object data may havebeen stored within the application 306 itself or may have been retrievedby the application 306 from the object database 318 or from anotherlocation such as an online location accessible via the Internet.Alternatively, the request to locate the specified object(s) includes,instead of the object data itself, data indicating a location from wherethe object data can be retrieved, e.g., from a location in the objectdatabase 318 or an online location. A name of the object or some otherdescription of the object is associated with the object data so as toallow the object data to be retrieved if necessary. Amongst otherthings, this name or description can also allow the input module 304 toindicate which one(s) of a set of different objects has been located.The request to locate the object(s) may also include other optionaldata, such as data indicating the maximum time period for attempting tolocate objects.

As used herein, the term object refers to a physical object or to a partof a physical object or to a plurality of physical objects. The termrefers equally to a class of such objects, i.e., any object in a set ofobjects having one or more similar characteristics. Examples of a classof objects would be ‘a flower’ or ‘a house’. It will be appreciated bypersons skilled in the relevant art that the object data includes datacharacterizing certain features of a specified object. The object datacan be used to recognize objects in the field of view of the camera 110,208 under various circumstances. For example, objects may be viewed fromdifferent angles, from different distances (and so have differentsizes), in different lighting conditions, and may also be partlyobscured. It will also be appreciated that objects having a range ofdifferent characteristics, e.g., objects having different shapes, colorsor textures or having fewer or more features, will nevertheless beconsidered as corresponding to the specified object. In some cases, theobject data is such that an object can still be recognized even if onlya part of the object is in the field of view of the camera 110, 208. Insome example embodiments, the object data is determined, e.g. duringdevelopment of the application 306, by way of an (offline) learningprocess involving analysis of a set of training images. In some exampleembodiments, the object data may include information relating to thethree-dimensional structure of the object. The form of the object datadepends upon the object recognition method used by the analyzing module312, which is described in more detail below.

After providing the request to locate the specified object(s) to theinput module 304, the application 306 may pause or may continue to carryout certain operations in the same time period in which the input module304 is also carrying out operations. For example, the application 306may continue to update the display of graphical information on a portionof the touchscreen 102 while a portion of the touchscreen 102 is alsobeing used by the input module 304. Alternatively, the application 306may continue to provide information to the user via the speaker 106.

At step S404, the input module 304 receives the request to locate thespecified object(s) and, if necessary, retrieves the object data from anindicated location in the object database 318 or from an onlinelocation, as described above. If the input module 304 is unable toretrieve the object data, then a response is sent to the application 306indicating that the object data cannot be retrieved.

Either the first camera 110 or the second camera 208 can be used in theinput method. However, the second camera 208 is more likely to be usedsince it generally provides higher quality image data and due to itsposition in relation to the touchscreen 102. The request to locate thespecified object(s) received from the application 306 may include dataindicating which of the cameras 110, 208 is to be used. If this data isnot provided, then the second camera 208 may be used by default. It willbe assumed hereinafter that the second camera 208 is to be used.

After receiving the request to locate the specified object(s), the inputmodule 304 activates the camera 208 if it is not already activated. Inother words, the input module 304 directs the camera 208 to beginsensing image data. Then, the input module 304 (see step S406) togetherwith the user interface module 302 (see step S408) cause the device 100to display the image data from the camera 208 on the touchscreen 102substantially in real time, thereby providing a camera viewfinder. Itwill be appreciated that the device 100 may also carry out otheroperations when the camera 208 is activated, such as, for example,autofocusing and automatically setting the sensitivity of the camerasensor. In some example embodiments, the user may also be able tocontrol the camera 208, for example to allow the focus or themagnification to be changed. This can be provided for by components ofthe input module 304 or by other software components (not shown) and bythe user interface software 302. Some of these operations are similar tothose carried out when a camera or camcorder application (not shown) isrunning. The input module 304 also starts a timer for timing the attemptto located the specified object(s).

Example screenshots as the user moves the device 100 in an attempt tolocate a specified object in the field of view of the camera 208 areshown in FIGS. 5B, 5C and 5D. Additional information may also bedisplayed on the touchscreen 102. This additional information caninclude, for example, the time elapsed since the attempt to locate theobject was started (as shown in FIGS. 5B, 5C and 5D), information aboutthe object(s) to be located, one or more user input buttons, and soforth.

As well as being displayed on the touchscreen 102, the image data fromthe camera 208 is also provided to the analyzing module 312 where it isanalyzed to determine whether or not a specified object is located inthe field of view of the camera 208 (see step S410). In some exampleembodiments, the analysis does not begin until a user input is received,e.g., via the image capture key 104 b or a button provided on thetouchscreen 102. This allows the user to have greater control over theobjects which are determined to be located. In some example embodiments,the user may be required to locate the object within a portion of thefield of view of the camera 208, which can be indicated by means of anoutline of a shape or other graphical information displayed on thetouchscreen 102. In this case, only image data within this portion ofthe field of view may be analyzed.

The image data may be subject to additional processing prior to beinganalyzed. For example, the resolution, color depth and frame rate of theimage data may be reduced.

The image data in each of a series of frames are analyzed using anobject recognition method to recognize specified object(s) as describedby the object data provided to the input module 304. Appropriate objectrecognition methods will be known per se to persons skilled in therelevant art. Relevant references include, inter alia:

-   1. D. Lowe, Object recognition from local scale-invariant features,    Proceedings of the Seventh IEEE International Conference on Computer    Vision, p. 1150 (1999).-   2. P. Viola and M. Jones, Rapid object detection using a boosted    cascade of simple features, Proceedings of the 2001 IEEE Computer    Society Conference on Computer Vision and Pattern Recognition (CVPR    2001), p. 511 (2001).-   3. V. Lepetit, P. Lagger, and P. Fua, Randomized trees for real-time    keypoint recognition, Proceedings of the 2005 IEEE Computer Society    Conference on Computer Vision and Pattern Recognition (CVPR    2005), p. 775 (2005).-   4. H. Bay, A. Ess, T. Tuytelaars and L. Van Gool, SURF: Speeded Up    Robust Features, Computer Vision and Image Understanding (CVIU),    vol. 110, no. 3, p. 346 (2008).

Analysis of the image data continues until either it is determined thatthe specified object(s) is located (see step S412) or that a certaintime period has elapsed since the attempt to locate the specifiedobject(s) was started (see step S414). The certain time period may beindicated by the application 306 in the request to locate the specifiedobject(s) or, if it is not indicated, a default value is used. The timeperiod may be of any appropriate duration which, in some cases, will bea few seconds and, in some other cases, a few minutes.

If the specified objects are not located before the certain time periodhas elapsed, then an indication that the objects have not been locatedis provided to the application 306 (see step S416). In response toreceiving this indication (see step S418), the application 306 resumesand carried out operations dependent upon the received indication (seestep S420). For example, the application 306 may give the user anotherchance to locate the object(s), instruct the user to locate a differentobject(s), provide some assistance to the user, or instruct the user toprovide an input via another means.

If at least one of the specified objects is located before the certaintime period has elapsed, then, in some example embodiments, the locatedobject(s) is highlighted by displaying graphical information on thetouchscreen 102. This is initiated by the input module 304 (see stepS422) and completed by the user interface software 302 (see step S424).For example, the object may be substantially enclosed by a shape such asa rectangle, as shown in FIG. 5D. Alternatively, the outline and/orinterior of the object(s) may be highlighted or an image or a symbol maybe displayed over the object or near to the object(s). An additionalfeedback signal, for example, a sound or vibration, may also be providedto the user when at least one of the specified objects is located.

In some example embodiments, the located object(s) is tracked, i.e., itsmovement within the field of view of the camera 208 is detected over aperiod of time. This can be performed by recognizing the specifiedobject(s) in each of a series of frames. Thus, the graphical informationhighlighting the object may be caused to move along with the locatedobject if the device 100 or the object moves. Moreover, in some exampleembodiments, a further timer can be started when the object(s) is firstlocated and the length of time during which the user is able to keep theobject(s) within the field of view can be measured. In other exampleembodiments, the touchscreen 102 displays a still image after thespecified object(s) has been located.

In cases where the user can select any one of a set of differentobjects, the user can be provided with a means for ignoring thehighlighted object, for example, by pressing the image capture key 104 bor a button provided on the touchscreen 102 within a relatively shorttime period of the object being first highlighted. If the user choosesto ignore the highlighted object, then the process returns to the stepof analyzing the image data (step S410). In cases where the user mustselect all of the objects in a set of different objects, if at least oneof the specified objects has been located and highlighted but one ormore other objects are still to be located, then the image data maycontinue to be analyzed to try to locate the one or more other objects.

In some example embodiments, the user can be provided with a means forstopping use of the input method and returning to the application 306,for example, by pressing the home key 104 a or a button provided on thetouchscreen 102

An indication that one or more of the specified object(s) has beenlocated is then provided to the application 306 (see step S426). Theindication may include data indicating, amongst other things, theidentity of the one or more objects which have been located, the timetaken to locate the object(s), or the length of time during which theobject was kept in the field of view. The indication may also includefurther information about the located object(s) such as, for example,information relating to color, aspect, size, inclination, etc. Thisfurther information can be determined by the analyzing module 312 eitherduring or after the object recognition.

In response to receiving the indication that the specified object(s) hasbeen located (see step S428), the application 306 resumes and carriesout operations dependent upon the received indication (see step S430).

For example, the application 306 can calculate a score based on the timetaken to locate the object(s), which one of a set of different objectshas been located or how many of a set of different objects have beenlocated. More specifically, the score can be inversely proportional tothe time taken or dependent upon the difficulty generally associatedwith locating certain objects or proportional to the number of objectslocated. The score can be displayed, together with other information, onthe touchscreen 102 as shown, for example, in FIG. 5E. The score canalso be combined with previous scores, used to determine subsequentoperations, stored in memory 214, 216, and/or shared via the Internet.

Also, the application 306 can determine its subsequent operations basedon which one of a set of different objects has been located. Forexample, the application 306 can carry out a certain set of operationsin response to receiving an indication that a certain object has beenselected and can carry out a another set of operations in response toreceiving an indication that another object has been selected. Inembodiments in which further details about the located objects are alsoincluded in the indication, then the application 306 can also determinewhich subsequent operations are to be carried out based on these furtherdetails. For example, the application 306 may use the further details todetermine the position of the user in relation to the object and thendirect the user to move accordingly. In some example embodiments, theapplication 306 may determine that another specified object is now to belocated and the process described above can be repeated for this otherspecified object. This may be performed while the camera viewfinder isstill being displayed, i.e., objects to be located may be generated ‘onthe fly’ based on objects located by the user.

When the application is a browser 306 c, some or all of the dataincluded in the indication received from the input module 304 may beprovided to a remote server which takes the appropriate action(s).

Thus, the herein described example embodiments can have the advantage ofproviding a way of allowing a user to control operations of the device100, for example to play games using the device 100. The user cancontrol operations by merely moving the device 100 (or objects) so as tolocate one or more specified objects in the field of view of the camera208 and so the user does not necessarily have to use input means such asthe touchscreen 102 or hardware keys 104 a, 104 b. Thus, the control canbe carried out more quickly and easily in some circumstances. The way ofcontrolling operations can also be used by applications 306 to provide aricher user experience and to provide a way of testing the visuo-motorskills of the user. It also allows an application 306 to interact withthe physical environment in a great variety of ways. For instance, theapplication 306 can implement a modern, technological implementation ofa traditional treasure hunt game

It should be realized that the foregoing example embodiments should notbe construed as limiting. Other variations and modifications will beapparent to persons skilled in the art upon reading the presentapplication. For instance, the device 100 may not have a touch sensitivedisplay 102, but instead may include a display and a numerical keypadand/or a QWERTY or equivalent keypad. The device 100 may also have onlyone camera or more than two cameras. Alternatively, the camera may beseparate from the device 100. Also, the input module 304 may be combinedwith the application 306. Here, functions said to be provided separatelyby the application 306 and the input module 304 are provided by theapplication 306. This may provide a smaller size of software needed toperform the required functions, although other applications may not beable to share the input module 304.

In some example embodiments, the user interface software 302 isintegrated with the input module 304. Here, functions said to beprovided separately by the user interface software 302 and the inputmodule 304 are provided by the combined user interface and inputsoftware module 304.

In other example embodiments, the input module 304 may be combined withthe user interface software 302 and the application 306. Here, functionssaid to be provided separately by the user interface software 302, theapplication 306 and the input module 304 are provided by the combinedsoftware.

Moreover, the disclosure of the present application should be understoodto include any novel features or any novel combination of featureseither explicitly or implicitly disclosed herein or any generalizationthereof and during the prosecution of the present application or of anyapplication derived therefrom, new claims may be formulated to cover anysuch features and/or combination of such features.

What is claimed is:
 1. A method comprising: causing a user interface toprovide an output directly or indirectly specifying at least one object;analyzing substantially real-time data from a camera to determine if theat least one object is located in the field of view of the camera; andin response to determining that the at least one object is located inthe field of view of the camera, causing an indication that the objecthas been located to be provided through the user interface.
 2. Themethod of claim 1, further comprising: causing display of thesubstantially real-time data from the camera.
 3. The method of claim 1,further comprising: in response to determining that the at least oneobject is located in the field of view of the camera, causing display ofgraphical information highlighting the at least one object.
 4. Themethod of claim 3, wherein the graphical information comprises theoutline of a shape such as a rectangle enclosing a major part of the atleast one object.
 5. The method of claim 1, further comprising:measuring an interval between providing the output directly orindirectly specifying at least one object and determining that the atleast one object is located in the field of view of the camera; andproviding a value calculated based on the measured interval.
 6. Themethod of claim 1, comprising: providing an output directly orindirectly specifying plural objects; analyzing the substantiallyreal-time data from the camera to determine if one of the plural objectsis located in the field of view of the camera; and in response todetermining that one of the plural objects is located in the field ofview of the camera, causing an indication of which of the plural objectshas been located to be provided through the user interface.
 7. Themethod of claim 6, further comprising: determining subsequent operationsof a process based on which of the plural objects has been located. 8.The method of claim 6, further comprising: providing a value calculatedbased on which of the plural objects has been located.
 9. The method ofclaim 1, wherein the method forms part of a game.
 10. An apparatus,comprising: at least one processor; and at least one memory includingcomputer program code; the at least one memory and the computer programcode configured to, with the at least one processor, cause the apparatusat least to perform: causing a user interface to provide an outputdirectly or indirectly specifying at least one object; analyzingsubstantially real-time data from a camera to determine if the at leastone object is located in the field of view of the camera; and inresponse to determining that the at least one object is located in thefield of view of the camera, causing an indication that the object hasbeen located to be provided through the user interface.
 11. Theapparatus of claim 10, wherein the at least one memory and the computerprogram code are configured to, with the at least one processor, causethe apparatus to perform: causing display of the substantially real-timedata from the camera.
 12. The apparatus of claim 10, wherein the atleast one memory and the computer program code are configured to, withthe at least one processor, cause the apparatus to perform: in responseto determining that the at least one object is located in the field ofview of the camera, causing display of graphical informationhighlighting the at least one object.
 13. The apparatus of claim 12,wherein the graphical information comprises the outline of a shape suchas a rectangle enclosing a major part of the at least one object. 14.The apparatus of claim 10, wherein the at least one memory and thecomputer program code are configured to, with the at least oneprocessor, cause the apparatus to perform: measuring an interval betweenproviding the output directly or indirectly specifying at least oneobject and determining that the at least one object is located in thefield of view of the camera; and providing a value calculated based onthe measured interval.
 15. The apparatus of claim 10, wherein the atleast one memory and the computer program code are configured to, withthe at least one processor, cause the apparatus to perform: providing anoutput directly or indirectly specifying plural objects; analyzing thesubstantially real-time data from the camera to determine if one of theplural objects is located in the field of view of the camera; and inresponse to determining that one of the plural objects is located in thefield of view of the camera, causing an indication of which of theplural objects has been located to be provided through the userinterface.
 16. The apparatus of claim 15, wherein the at least onememory and the computer program code are configured to, with the atleast one processor, cause the apparatus to perform: determiningsubsequent operations of a process based on which of the plural objectshas been located.
 17. The apparatus of claim 15, wherein the at leastone memory and the computer program code are configured to, with the atleast one processor, cause the apparatus to perform: providing a valuecalculated based on which of the plural objects has been located. 18.The apparatus of claim 10, wherein the at least one memory and thecomputer program code are configured to, with the at least oneprocessor, cause the apparatus to perform a method which form part of agame.
 19. The apparatus of claim 10, wherein the apparatus is a devicefurther comprising: user interface circuitry and user interface softwareconfigured to facilitate user control of at least some functions of thedevice though use of a display and configured to respond to user inputs;and a display and display circuitry configured to display at least aportion of a user interface of the device, the display and displaycircuitry configured to facilitate user control of at least somefunctions of the device.
 20. A non-transitory computer readable mediumhaving stored thereon instructions for causing a computing apparatus toperform a method comprising: causing a user interface to provide anoutput directly or indirectly specifying at least one object; analyzingsubstantially real-time data from a camera to determine if the at leastone object is located in the field of view of the camera; and inresponse to determining that the at least one object is located in thefield of view of the camera, causing an indication that the object hasbeen located to be provided through the user interface.